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FOREWORD 


C^y( 

This  is  a  Drjef  report  of  a  literature  search  to 
establish  whether  os-noi^ combat  data  obtained  from  World  War  II, 
the  Korean  War,  and  the  Vietnam  War  show  promise  of  establishing 
the  existence  of  a  learning  effect  in  pilots  who  are  continually 
engaged  in  air-to-air  combat  over  a  period  of  time.  Mr.  Ralph  C. 

Lenz  of_ASB.  requested  the.  J.nYestigatj.on . _ 

^  The  study  reported  is  a  preliminary  effort,  and  the  evidence 
thus  far  is  inconclusive  regarding  the  existence  of  a  learning 
effect.  Thus,  a  more  detailed  follow-on  analysis  seems  justified. 
It  will  necessitate  further  literature  search  and  statistical 
analysis  of  moro  data. 
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I .  Introduction 


definitive  study  to  establish  if  a  learning  effect  exists 
aaong  pilots  who  routinely  engage  in  air-to-air  combat  must  be 
based  on  statistical  analyses  of  combat  data.  Fundamental  to  any 
statistical  analysis  is  an  adequate  sample  size  of  data,  with  the 
data  in  a  fora  which  is  appropriate  for  the  hypothesis  being 
tested. _ x 

A  literature  search  for  data  is  therefore  the  fundamental 
step  in  the  study  we  are  addressing.  Thus  far,  the  literature 
search  has  resulted  in  data  which  is  generally  inadequate,  for  one 
reason  or  another.  Hie  main  source  of  valid  data  and  results  which 
we  presently  have  is  a  paper  by  II.  Weiss  (Reference  6).  The 
bibliography  at  the  end  of  Weiss'  paper  is  a  potential  source  of 
literature,  but  some  of  these  references  have  been  impossible  to 
obtain  up  to  the  present  time. 

II.  Discussion  and  Results 

Some  early  (World  War  II)  attempts  were  made  to  quantify 
aerial  warfare  (7),  but  these  references  contained  little  on  pilot 
performance  and  were  concerned  mostly  with  flight  dynamics. 

R.  F.  Putrell  and  the  staff  of  USAF  Historical  Division,  Air 
University,  have  compiled  volumes  on  USAF  actions  in  Korea  (1), 

(2).  These  volumes  outline  the  number  of  air  battles  in  the  Korean 
conflict  and  detail  some  data  on  combat  aces.  In  particular  it  was 
shown  that  68%  of  the  pilots  who  destroyed  MIGs  were  over  28  years 
old,  and  that  this  was  significantly  greater  than  the  age  of  the 
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overall  population  of  combat  pilots.  It  was  also  shown,  by 

Mr.  Futrell,  that  success  and  failure  in  air-to-air  combat  occurred 

in  grouped  events,  with  many  inactive  sorties  between. 

.  The  Korean  Air  War  Summary  (3)  shows  a  total  of  14,916  air- 
to-air  encounters  with  almost  a  7:1  (900/139)  USAF-to-Red  kill 
ratio.  Data  on  individual  pilot  experience  is  not  available,  however, 
in  a  span  of  13  months,  the  ratio  of  aircraft  killed  (Red/US AF) 
showed  a  marked  increase  from  6/1  to  16/1  in  spite  of  a  small 
initial  decrease.  (See  Table  1)  Applying  a  statistical  trend  test  (4) 
to  the  data,  a  significant  trend  toward  increasing  USAF  superiority 
existed,  litis  test  however,  does  not  analyze  individual  duels  for 
pilot  survival,  since  data  arc  not  available  on  previous  pilot 
conflicts. 

SEA  data  is  contained  in  the  Red  Baron  statistics.  These 
statistics  have  not  been  fully  analyzed  by  the  Red  Baron  group  at 
Nellis  AFB  nor  the  WSEG/IDA  group  in  Washington,  DC.  Results  of 
individual  air-to-air  decisive  encounters  are  known  but  the  history 
of  the  pilot's  previous  encounters  has  not  been  published  (S). 

To  determine  pilot  survival  after  a  number  of  encounters  will 
require  reviewing  the  raw  data  at  WSEG/IDA  and  at  Nellis  AFB.  A 
sample  of  a  raw  data  sheet  is  presented  as  Appendix  B.  The  number 
of  encounters  experienced  in  SEA  is  small  compared  to  the  Korean 
data;  the  clear  trend  of  increasing  superiority  established  in 
the  Korean  War  has  not  yet  had  a  chance  to  be  established  in  SEA. 

II.  Weiss  (6)  has  written  a  paper  which  includes  some  statistics 
on  pilot  survival  in  air-to-air  combat.  Weiss  finds  from  WW  I 


2 


TABLE  1 


USAF  Claims/Kills  and  Losses  In 
Korean  War  Air- to- Air  Combat* 


Time  Period 

Enemy  Aircraft 
Destroyed 

USAF  Aircraft 
Destroyed 

Rounded  Ratio 
Enemy  Dest/ 
USAF  Dest 

July-Sept  1950 

32 

5 

6/1 

Oct-Dec  1950 

17 

4 

4/1 

Jan-March  1951 

17 

4 

4/1 

Apr-June  1951 

40 

11 

*/r 

July-Sept  1951 

30 

10 

3/1 

Oct-Dec  1951 

88 

31 

3/1 

Jan-March  1952 

87 

13 

7/1 

Apr-June  1952 

97 

18 

5/1 

July-Sept  1952 

116 

15 

8/1 

Oct-Dec  1952 

83 

13 

6/1 

Jan-March  1952 

98 

8 

12/1 

Apr-June  1952 

163 

5 

33/1 

July  1953 

32 

2 

16/1 

Total 

900 

139 

7/1 

*  Adapted  from  "Korean  Air  War  Summary:  Organizations  Under  the 
Operational  Control  of  FEAT,  25  June  1950  -  27  July  1953" 
Archives  Pile  No:  K720.04D-1. 


American  and  French,  WW  I  German,  and  WW  II  German  data  the 
probability  of  a  pilot  being  killed  on  the  j-th  decisive  encounter. 

A  decisive  encounter  is  defined  as  an  air-to-air  encounter  where 

one  of  the  opponents  is  downed.  The  probability  curve  (Figure  1) 

shows  a  curvilinear  decrease  in  the  probability  of  being  killed 

as  the  number  of  encounters  increases,  as  would  be  expected 

when  a  learning  effect  exists.  The  curve  levels  off  at  a  probability 

of  0.02  after  about  ten  encounters. 

Next,  H.  Weiss  proceeds  to  hypothesize  "survival  of  the  fittest" 
as  the  reason  for  the  decline  in  attrition.  To  support  this  view 
he  calculates  the  distribution  of  "probability  of  survival"  from 
each  encounter,  assuming  the  probability  of  survival  will  be  drawn 
from  a  stationary  distribution  at  each  encounter.  From  the  data, 
the  distribution  from  which  the  chance  of  survival  is  drawn  at 
' each  encountor  was  calculated  3nd  has  a  U  shape  (Figure  2) ,  or  a 
bimodel  form.  This  indicates  that  pilots  are  either  skilled  or 
relatively  unskilled,  few  pilots  being  of  moderate  skill.  The 
calculation  of  the  U-shaped  distribution  is  a  result  of  the  hypo¬ 
thesis  that  "the  survival  of  the  fittest"  is  the  phenomenon  which 
is  present.  In  other  words,  the  U-shaped  curve  is  a  result  of  an 
hypothesis,  it  is  not  additional  evidence  of  the  premise  of 
"survival  of  the  fittest." 

One  could  argue  that  the  Korean  War  data,  which  shows  a 
trend  towards  increasing  USAF  superiority,  would  support  the 
hypothesis  that  a  learning  effect  is  present.  However,  one  could 
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also  have  such  a  trend  if  '{survival  of  the  fittest"  were  the  pre¬ 
dominate  effect.  Such  an  effect  would  cause  an  enrichment  of  the 
population  of  good  pilots  as  time  elapsed  which  would  mean  that 
combat  scores  of  the  pilot  population  as  a  whole  would  get  better. 

To  fully  test  the  hypothesis  of  a  learning  effect  in  air-to- 
air  combat  will  probably  involve  a  two- fold  approach.  One  would 
be  analyses  of  combat  data  on  individual  pilots  and  the  other  would 
be  to  test  for  a  learning  effect  under  controlled  experimental  . 
conditions.  This  latter  approach  is  necessary  in  order  to  have 
subjects  remain  "alive"  to  determine  if  they  gained  experience 
from  the  previous  encounter. 

One’s  intuition  suggests  that  probably  both  effects,  survival- 
of-the-fittest  due  to  innate  skills  and  learning  by  exposure 
to  repeated  combats,  are  present.  There  is  also  a  third  effect 
which  comes  to  mind,  and  that  is,  the  interaction  with  enemy 
skill  level,  and  whether  that  skill  level  changes  with  time.  This 
additional  consideration  is  an  argument  against  relying  exclusively 
on  combat  data  to  test  the  hypothesis  of  learning,  because  there 
are  so  many  uncontrolled  factors  present;  one  does  not  really 
know  what  is  causing  the  variations  in  fraction  of  pilots 
surviving.  Controlled  laboratory  conditions,  i.e.,  simulation,  may 
be  the  only  answer. 

This  has  been  a  preliminary  report.  We  propose  to  follow 
this  with  a  more  detailed  and  rigorous  presentation  on  the  short¬ 
comings  of  analyses  of  combat  data,  even  if  such  data  were 
available,  and  a  proposed  experimental  approach  to  testing  the 
learning  hypothesis. 


PROBABILITY  OF  BEING  KILLED 


PROBABILITY  DENSITY 


APPENDIX  A 

Extracts  from  the  Paper 

by  11.  Weiss 


Air-to-Air  Cbmbat 

Aircraft  are  loat  to  enemy  aircraft  ae  well 
ae  to  flak.  Data  on  past  air  combat  wai  therefore 
'examined  to  determine  a  bati*  for  a  model  of  this 
phase  of  air  operations.  In  attempting  to  develop 
an  analytical  model  to  represent  the  Battle  of 
Britain,  the  author  found  that  there  was  a  strong 
indication  that  each  side  lost  aircraft  in  proportion 
to  the  number  committed  to  the  air  battle,  rela¬ 
tively  independently  of  the  number  of  enemy  air¬ 
craft  present.  At  the  same  time  it  was  noted  that 
in  all  past  wars  involving  extensive  air-to-air  com¬ 
bat,  a  small  number  of  pilots  —the  aces  —  were  re¬ 
sponsible  for  most  of  the  kills.  It  was  therefore 
hypothesized  that  fighter  force  capability  depended 
on  the  performance  of  a  few  top  pilots  rather  than 
numbers  of  pilots  and  attention  was  shifted  to  mea¬ 
sures  of  pilot  performance.  The  following  routine 
was  employed  to  obtain  a  measure  of  pitot 
effectiveness: 

A  "decisive  combat"  is  defined  as  one  in 
which  a  pilot  is  either  killed  or  adds  one  to  his 
score.  •  (It  is  recognized  that  the  method  is  depen¬ 
dent  on  consistency  of  the  scoring  system  and  the 
results  depend  on  the  mix  of  enemy  aircraft  types.) 


Let 

T.  *  total  number  of  pilots,  living  and 
*  dead  with  score  "j" 

K.  :  number  of  pilots  KIA  by  enemy 
J  aircraft  with  score  "j" 

R.  *  number  of  pilots  leaving  combat, 

J  other  than  KIA  in  air  combat, 

with  score  "j" 

C.  *  number  of  pilots  entering  their 
^  jth  decisive  combat 

p.  *  probability  that  a  pilot  will  be 
^  killed  in  his  jth  decisive  combat 

S.  *  £  T  *  total  number  of  pilots 

J  saj  living  or  dead  with  at  (24) 

least  score  j 

and 

VKj-i/(VKj-i>  '  <«> 


Then  the  flow  diagram  of  Figure  13  traces  the  pro¬ 
gress  of  a  pilot  from  his  first  combat  through  hie 


■IMHDOS 
UANWtUtB 
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Figure  13.  Tighter  Pilot  Activity  now 


Although  a  moderate  amount  of  information 
is  available  on  Aces,  18,  19  d»ta  was  located  on 
only  three  organizations  which  permitted  compu¬ 
tation  of  pj  for  pilots  with  scores  of  1  to  4.  These 
were  Richthofen’s  Jagdgeschwader  Nr.  1,20  and 
American  pilots  serving  with  the  French  (including 
the  Lafayette  Escadrille)*** 22  in  World  War  I, 
and  Jagdgeschwader  JG  26  in  World  War  Ij23  for 
these  three  organizations,  p.  is  plotted  against 
score  in  Figure  14.  J 

The  initial  almost  vertical  drop  in  proba¬ 
bility  of  being  killed  between  decisive  combats  one 
aiid  five  was  completely  unexpected. 

The  v.alue  of  about  0.  02  in  the  range  *10-30 
is  consistent  with  similarly  computed  values  for 
American  Aces  in  World  War  II  and  Korea,  which 
lell  in  the  range  0.  01  to  0.  03. 
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Figure  14.  Fighter  Pilot  Loee  Rete  vereue  Score 


The  question  immediately  arises  whether 
the  initial  decline  in  p  with  score  represents  learn¬ 
ing,  or  the  elimination  of  the  least  skilled  pilots. 
An  improvement  factor  of  twenty  in  five  "trials  by 
combat"  seems  less  likely  to  the  writer  than  the 
hypothesis  that  Figure  14  represents  the  survival 
of  the  fittest. 

Since  a  pilot's  "score"  includes  reconnais¬ 
sance  aircraft  and  bombers  as  well  as  fighters,  the 
records  were  examined  to  see  whether  the  high 
scores  were  based  largely  on  "sitting  ducks." 

This  was  found  not  to  be  the  case;  a  fair  estimate 
appears  to  be  that  on  the  average  a  pilot's  score 
contains  both  fighters  and  other  aircraft  in  fairly 
equivalent  numbers.  If  non-fighter  aircraft  are 
easier  targets,  therefore,  the  descent  of  the  "p" 
curve  in  a  pure  fighter  environment  would  be  even 
steeper,  by  a  factor  of  perhaps  2.  0. 

The  following  analysis  was  therefore  per¬ 
formed.  It  was  assumed  that  the  capability  of  a 
pilot  entering  combat  could  be  represented  by  a 
value  "s,"  the  probability  that  he  would  survive  a 
decisive  combat,  and  that  "s"  characterized  his 
skill  and  changed  insignificantly  in  successive  com¬ 
bats.  (A  modified  model  might  of  course  allow  for 
some  learning.)  The  fraction  of  pilots  of  capability 
"a"  is  described  by  the  probability  density  function 
f(s).  Between  decisive  combats  if  was  assumed 
that  all  pilots  regardless  of  skill,  had  an  equal 
probability  of  leaving  combat.  Define 

Vj  *  J  s^  f(s)  ds  (26) 


I. e.  this  is  the  expected  fraction  of  the  initialforce 
surviving  "j"  decisive  combats.  Subject  to  the  addi¬ 
tional  assumption  of  equal  probabilities  of  with¬ 
drawal  between  coinbats, 

Pj  “  *  *  <Z7> 

hence 

Vj  *  "  (28) 

and  Vi  can  be  compute'd  from  the  data.  Since  the 
Vj  are  the  moments  of  the  distribution  f(s),  f( ») 

,y  be  computed  from  them  and  this  has  beendone. 
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Figure  15  ahowe  ((a)  plotted  against  proba¬ 
bility  of  being  killed,  (1-a).  The  U- shaped  distribu- 
(ion  ia  aurprialng—  there  aeem  to  be  few  "average" 
(jghter  pilota.  Again,  an  initial  conjecture  that  the 
distribution  might  be  normal,  turned  out  to  be 
unaupportable.  Figure  16  which  cumulate  a  the 
probability  denaity  function  ahowa  that  for  thiadata, 
at  beat  fewer  than  IS  percent  of  the  pilota  had  a 
better  than  even  chance  of  eurviving  their  firat 
combat. 


MwtneiMnwHVaMiawt' 


Figure  IS,  Probability  Denaity  Function  of 
Pilot  Performance 


ftoaaaairvpxir  king  ratio  n  Txcatvt  combat' 


Figure  IS.  Cumulative  Dlatrlbutlon  Function  of 
Pilot  Performance 

Next  examining  the  acorea  of  the  top  ten 
tighter  pilota  of  aeveral  countries  in  both  world 
.  *'**••  Figure  17  ahowa  the  tremendous  contribution 
#  VU?"  number  of  men.  The  high  reportedacore 
ol  the  German  Pilots^*  (which  will  undoubtedly  be 
the  subject  argument  ad  infinitum)  becomea  plau- 
'* J‘"  one  comparea  their  values  of  "p"  and 

a  r*te-  ln  Figure  IS.  With  "p"  of  0.  01  to 

'*•  Wffittl  of  all  Acea  of  all  countriea)  and  no 
g.  tlat*  ****  o*P«cled  number  of  killa  per  Ace 

*•  allot  down  la  SO  to  100,  and  lndivlduata 
"  of  aeveral  hundred  are  not  unexpected. 
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Figure  17. 

Total  Scores  of  Top  Ten  Acer 
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20 

Figure  18.  'Comparison  of  Single  Combat 
bon  Rate  and  Cumulative 

A  achematic  model  of  air-to-air  combat  now 
suggests  itself.  At  the  same  time,  it  is  interesting 
to  consider  the  effects  of  a  policy  which  uses  com¬ 
bat  outcome  as  a  basis  for  upgrading  average  pilot 
skill.  For  this  paper  it  is  assumed  that  both  forces 
operate  from  sanctuary  bases,  that  onlyfightcra  are 
involved,  and  that  one  side  attempts  to  rescue  its 
pilots  who  survive  the  loss  of  their  aircraft.  The 
model  is  diagramed  in  Figure  19. 


OTnaroaa  com— T  M»  KX] 


Figure  19.  Air-Air  Combat  Activity  Flow 


It  is  further  assumed  that  there  are  only  two 
classesof  pilots,  "Hawks"  and  "Doves,  "  that  Hawks 
represent  10  percent  of  replacement  pilots  and  can¬ 
not  be  identified  before  combat.  Combat  takes  place 
between  two  aircraft  at  a  time;  the  probability  that 
aach  combat  involves  Hawks.  Doves,  or  one  of 
each,  is  proportional  to  their  representation  in  the 
individual  forces.  Hawks  always  ahoot  down  Dovea, 
and  a  Hawk  baa  an  even  chance  againat  an  enemy 
Hawk.  Combat  between  Dovea  reaulta  in  no  loaa. 
Forcea  on  both  aidea’ara  equal,  and  aircraft  and 
pilots  are  replaced  aa  they  areloat.  Pilota  may  sur¬ 
vive  the  loaa  of  their  aircraft  with  specified  proba¬ 
bilities.  (Soma  German  Acea  ofWorld  War  Uara  re¬ 
ported  to  have  been  shot  down  six  times  er  more 
and  survived. ) 


Rescuing  pilots  recovers  the  investment  in 
t reining,  but  does  not  improve  force  effectiveness 
drnsticslly,  unless  coupled  with  s  selection  pro¬ 
cess.  It  is  therefore  further  Assumed  that  each 
rescued  pilot  with  a  score  of  one  or  more  returns 
to  combat;  but  that  rescued  pilots  shot  down  with¬ 
out  score  are  transferred  to  noncombat  flying 
duties. 

The  result  of  this  selection  process  is  a 
substantial  increase  in  the  effectiveness  of  the 
force  employing  it,  with  Figure  20  showing  the 
results.  If  80  percent  of  pilots  survive  the  loss  of 
their  aircraft  and  all  are  rescued,  with  those  hav¬ 
ing  prior  scores  returning  to  the  combat,  the  force 
effectiveness  is  tripled  in  a  sustained  combat. 


Figure  20.  Effect  of  Rescue  Plus  Selection 
Doctrine  on  Force  Effectiveness 


Figure  II.  effect  ml  Rescue  Plus  Selection  . 

Doctrine  on  Pilot  Replacement  Rate 


Figure  21  shows  the  required  pitot  replacement 
rate.  Figure  21  may  be  misleading,  the  new  pilots 
required  per  downed  enemy  aircraft  depends  only 
on  tae  rescue  rate,  to  a  first  approximation. 

On  the  other  hand,  if  the  precombat  train¬ 
ing  and  screening  process  delivers  only  "Hawks" 
to  one  side,  that  side  may  have  a  10:1  sustained 
exchange  ratio,  at  all  times. 

Discussion  of  Air-to-Air  Combat 

The  foregoing  analysis  and  model  has  said 
nothing  about  equipment  characteristics.  It  is 
clear  that  both  equipment  and  men  are  vital.  Pro¬ 
longed  major  wars  in  the  past  have  tended  to  wit- 
nest  the  development  of  aircraft  of  compatible  per¬ 
formance  on  both  sides.  In  all  wars  these  differ¬ 
ences  have  been  far  overshadowed  by  the  perfor¬ 
mance  of  Aces,  as  individuals. 

Before  Korea  it  was  believed  thatair-to-air 
combat  between  fighter  aircraft  was  obsolescent, 
or  would  be  combat  between  machines,  gun  sights, 
and  computers.  Events  turned  out  otherwise.  The 
writer  suggests  that  the  increasing  complexity  of 
equipment,  and  the  incredibly  demanding  environ¬ 
ment  of  air  combat  will  only  reduce  to  even  smaller 
numbers,  those  individuals  who  can  master  their 
equipment  and  the  combat  environment,  and  whose 
.presence  as  dozens,  within  a  force  of  hundreds,  or 
thousands,  will  be  decisive. 

It  seems  clear,  in  addition,  that  any  real¬ 
istic  assessment  of  the  capabilities  of  projected 
equipment  must  properly  account  for  the  variability 
of  individual  performance,  and  allow  the  selection 
and  maximum  exploitation  of  the  rare  capabilities 
of  the  best  operators,  while  raising  to  a  maximum 
the  performance  of  the  less  skilled.  Conversely  an 
attempt  to  assess  the  performance  of  equipment 
must  correct  for  the  variability  of  the  humans  who 
operate  it. 


Conclusion 

This  paper  has  proceeded  from  a  broad  dis¬ 
cussion  of  the  objectives  of  system  analysis  through 
the  outlining  of  the  structure  of  a  large  and  complex 
operation  to' the  development  of  specific  submodels 
and  their  interrelationships  to  combat  data,  and  to 
system  performance  parameters. 

In  War  and  Peace  Tolstoy  had  Prince  Andrew 
remark: 

"What  theory  and  science  is  possible 
about  a  matter  the  conditions  and 
circumstances  of  which  are  unknown 
and  cannot  be  defined,  especially 
when  the  strength  of  the  acting  forces 
cannot  be  ascertained?  . . .  What 
science  can  there  be  in  a  matter  in 
which,  as  in  all  practical  matters, 
nothing  can  be  defined  and  everything 
depends  on  innumerable  conditions, 
the  significance  of  which  is  deter¬ 
mined  at  a  particular  moment  which 
arrives  no  one  knows  when?" 
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But  the  experience  of  the  pelt  half  century 
is  that  more  can  be  known  about  the  "calculus  of 
conflict"  than  was  envisioned  by  the  Prince.  The 
analyst  is  always  subject  to  the  overriding  judgment 
of  military  experience,  but  increasingly  that  experi¬ 
ence  and  judgment  are  susceptible  to  expression  in 
quantitative  terms. 


The  methodology  now  exists  for  producing 
analytical  tools'  of  convincing  verisimilitude- but 
both  the  analyst  and  the  military  user  mushcontinue 
to  remain  aware  of  the  fact  that  this  appearance  of 
truth  may  be  false,  that  the  validity  of  an  analysis 
is  subject  to  proof  in  the  "moment  of  truth"  on  the 
battlefield,  that  verification  is  of  such  importance 
that  all  possible  avenues  of  test  from  field  exer¬ 
cises  to  combat  records  must  be  utilised. 
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APPENDIX  B 

Saaple  Data  Sheet 
Project  Red  Baron 


C/J  O 


RED  BARON 

MIG  INCIDENT  SUMMARY 


I.  BASIC  DATA  Event  No.  _ 

Date/Time  _ 

Service/Unit  _ 

No.  &  Type  A/C: 

U.S. 

Enemy 


II.  MISSION  DATA 


Mission 

Location 

1  Kill/Damage 

I  Ordnance 

of  Encounter 

U.S. 

Enemy 

■ 

-■t.  • 

III.  PILOT  DATA 

Plight 
Pos.  & 
all 
lgn 


Name.  Rank.  S/N 


Dwn/ 

Drcg 


Weapons 

Fired 


Enemy 

K/D 


Present 

Location 


Inter¬ 

viewed 

Date/By 


IV.  IDENTIPYINO  COMMENTS 


V.  NOTES 
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